In this work, we propose a semantic flow-guided two-stage framework for shape-aware face swapping, namely FlowFace. Unlike most previous methods that focus on transferring the source inner facial features but neglect facial contours, our FlowFace can transfer both of them to a target face, thus leading to more realistic face swapping. Concretely, our FlowFace consists of a face reshaping network and a face swapping network. The face reshaping network addresses the shape outline differences between the source and target faces. It first estimates a semantic flow (i.e., face shape differences) between the source and the target face, and then explicitly warps the target face shape with the estimated semantic flow. After reshaping, the face swapping network generates inner facial features that exhibit the identity of the source face. We employ a pre-trained face masked autoencoder (MAE) to extract facial features from both the source face and the target face. In contrast to previous methods that use identity embedding to preserve identity information, the features extracted by our encoder can better capture facial appearances and identity information. Then, we develop a cross-attention fusion module to adaptively fuse inner facial features from the source face with the target facial attributes, thus leading to better identity preservation. Extensive quantitative and qualitative experiments on in-the-wild faces demonstrate that our FlowFace outperforms the state-of-the-art significantly.
translated by 谷歌翻译
大多数图形之间的作品都是在具有交叉注意机制的编码器框架上构建的。最近的研究表明,对输入图结构进行明确建模可以显着改善性能。但是,香草结构编码器无法在所有解码步骤的单个正向通道中捕获所有专业信息,从而导致语义表示不准确。同时,输入图在交叉注意中作为无序序列被扁平,忽略了原始图形结构。结果,解码器中获得的输入图上下文向量可能存在缺陷。为了解决这些问题,我们提出了一种结构感知的交叉注意(SACA)机制,以在每个解码步骤中以结构意识的方式重新编码在新生成的上下文上的输入图表示条件。我们进一步调整SACA,并引入其变体动态图修剪(DGP)机制,以在解码过程中动态下降无关的节点。我们在两个图形数据集(LDC2020T02和ENT-DESC)上实现了新的最新结果,但计算成本仅略有增加。
translated by 谷歌翻译
基于步态阶段的控制是步行AID机器人的热门研究主题,尤其是机器人下限假体。步态阶段估计是基于步态阶段控制的挑战。先前的研究使用了人类大腿角的整合或差异来估计步态阶段,但是累积的测量误差和噪声可能会影响估计结果。在本文中,提出了一种更健壮的步态相估计方法,使用各种运动模式的分段单调步态相位大角模型的统一形式。步态相仅根据大腿角度估算,这是一个稳定的变量,避免了相位漂移。基于卡尔曼滤波器的平滑液旨在进一步抑制估计步态阶段的突变。基于提出的步态相估计方法,基于步态阶段的关节角跟踪控制器是为跨股骨假体设计的。提出的步态估计方法,步态相和控制器通过在各种运动模式下的步行数据进行离线分析来评估。基于步态阶段的控制器的实时性能在经际假体的实验中得到了验证。
translated by 谷歌翻译
深层神经网络预测交通需求已引起了学术界和行业社会的广泛兴趣。其中,成对来源点(OD)需求预测是一个有价值但具有挑战性的问题:(i)大量可能的OD对,(ii)空间依赖性的内在性和(iii)交通的复杂性状态。为了解决上述问题,本文提出了一种连续的时间和多级动态图表表示方法,用于原始用途需求预测(CMOD)。首先,构建了一个连续的动态图表示学习框架,该框架维护每个流量节点(地铁站或出租车区)的动态状态向量。国家向量保留历史交易信息,并根据最近发生的交易不断更新。其次,提出了多层结构学习模块,以模拟站点级节点的空间依赖性。它不仅可以从数据自适应地利用节点之间的关系,还可以通过集群级别和区域级虚拟节点共享消息和表示形式。最后,跨级融合模块旨在集成多级记忆并为最终预测生成综合节点表示。在北京地铁和纽约出租车的两个现实世界数据集上进行了广泛的实验,结果证明了我们的模型与最先进的方法相比。
translated by 谷歌翻译
改善深度神经网络(DNN)对抗对抗示例的鲁棒性是安全深度学习的重要而挑战性问题。跨越现有的防御技术,具有预计梯度体面(PGD)的对抗培训是最有效的。对手训练通过最大化分类丢失,通过最大限度地减少从内在最大化生成的逆势示例的丢失来解决\ excepitient {内部最大化}生成侵略性示例的初始最大优化问题。 。因此,衡量内部最大化的衡量标准是如何对对抗性培训至关重要的。在本文中,我们提出了这种标准,即限制优化(FOSC)的一阶静止条件,以定量评估内部最大化中发现的对抗性实例的收敛质量。通过FOSC,我们发现,为了确保更好的稳健性,必须在培训的\ Texit {稍后的阶段}中具有更好的收敛质量的对抗性示例。然而,在早期阶段,高收敛质量的对抗例子不是必需的,甚至可能导致稳健性差。基于这些观察,我们提出了一种\ Texit {动态}培训策略,逐步提高产生的对抗性实例的收敛质量,这显着提高了对抗性培训的鲁棒性。我们的理论和经验结果表明了该方法的有效性。
translated by 谷歌翻译
随着文献资源的丰富,研究人员面临着信息爆炸和知识过载的不断增长的问题。为了帮助学者检索文学并成功获得知识,澄清学术文学中内容的语义结构已成为基本的研究问题。在识别学术文章中章节的结构功能的研究中,只有几项研究使用了深度学习模型,并探索了特征输入的优化。这限制了研究任务深度学习模型的应用,优化潜力。本文将ACL会议的文章作为语料库。我们采用传统的机器学习模型和深度学习模型,基于各种特征输入构建分类器。实验结果表明,(1)与章节内容相比,章节标题更有利于识别学术文章的结构功能。 (2)相对位置是建立传统模型的有价值的功能。 (3)受到(2)的启发,本文进一步将上下文信息引入深度学习模型,取得了显着的结果。同时,我们的模型在包含200个采样的非训练样本的开放式测试中显示出良好的迁移能力。近五年我们还基于表演模型的最佳实践,并对整体语料库进行了时间序列分析,近五年注释了ACL主要会议文件。这项工作通过多个比较实验探索并总结了此任务的实际功能和模型,并为相关文本分类任务提供了参考。最后,我们表示当前模型的局限性和缺点以及进一步优化的方向。
translated by 谷歌翻译
We introduce Argoverse 2 (AV2) - a collection of three datasets for perception and forecasting research in the self-driving domain. The annotated Sensor Dataset contains 1,000 sequences of multimodal data, encompassing high-resolution imagery from seven ring cameras, and two stereo cameras in addition to lidar point clouds, and 6-DOF map-aligned pose. Sequences contain 3D cuboid annotations for 26 object categories, all of which are sufficiently-sampled to support training and evaluation of 3D perception models. The Lidar Dataset contains 20,000 sequences of unlabeled lidar point clouds and map-aligned pose. This dataset is the largest ever collection of lidar sensor data and supports self-supervised learning and the emerging task of point cloud forecasting. Finally, the Motion Forecasting Dataset contains 250,000 scenarios mined for interesting and challenging interactions between the autonomous vehicle and other actors in each local scene. Models are tasked with the prediction of future motion for "scored actors" in each scenario and are provided with track histories that capture object location, heading, velocity, and category. In all three datasets, each scenario contains its own HD Map with 3D lane and crosswalk geometry - sourced from data captured in six distinct cities. We believe these datasets will support new and existing machine learning research problems in ways that existing datasets do not. All datasets are released under the CC BY-NC-SA 4.0 license.
translated by 谷歌翻译
Learning the underlying distribution of molecular graphs and generating high-fidelity samples is a fundamental research problem in drug discovery and material science. However, accurately modeling distribution and rapidly generating novel molecular graphs remain crucial and challenging goals. To accomplish these goals, we propose a novel Conditional Diffusion model based on discrete Graph Structures (CDGS) for molecular graph generation. Specifically, we construct a forward graph diffusion process on both graph structures and inherent features through stochastic differential equations (SDE) and derive discrete graph structures as the condition for reverse generative processes. We present a specialized hybrid graph noise prediction model that extracts the global context and the local node-edge dependency from intermediate graph states. We further utilize ordinary differential equation (ODE) solvers for efficient graph sampling, based on the semi-linear structure of the probability flow ODE. Experiments on diverse datasets validate the effectiveness of our framework. Particularly, the proposed method still generates high-quality molecular graphs in a limited number of steps.
translated by 谷歌翻译
Stance detection refers to the task of extracting the standpoint (Favor, Against or Neither) towards a target in given texts. Such research gains increasing attention with the proliferation of social media contents. The conventional framework of handling stance detection is converting it into text classification tasks. Deep learning models have already replaced rule-based models and traditional machine learning models in solving such problems. Current deep neural networks are facing two main challenges which are insufficient labeled data and information in social media posts and the unexplainable nature of deep learning models. A new pre-trained language model chatGPT was launched on Nov 30, 2022. For the stance detection tasks, our experiments show that ChatGPT can achieve SOTA or similar performance for commonly used datasets including SemEval-2016 and P-Stance. At the same time, ChatGPT can provide explanation for its own prediction, which is beyond the capability of any existing model. The explanations for the cases it cannot provide classification results are especially useful. ChatGPT has the potential to be the best AI model for stance detection tasks in NLP, or at least change the research paradigm of this field. ChatGPT also opens up the possibility of building explanatory AI for stance detection.
translated by 谷歌翻译
In this paper, a Kinect-based distributed and real-time motion capture system is developed. A trigonometric method is applied to calculate the relative position of Kinect v2 sensors with a calibration wand and register the sensors' positions automatically. By combining results from multiple sensors with a nonlinear least square method, the accuracy of the motion capture is optimized. Moreover, to exclude inaccurate results from sensors, a computational geometry is applied in the occlusion approach, which discovers occluded joint data. The synchronization approach is based on an NTP protocol that synchronizes the time between the clocks of a server and clients dynamically, ensuring that the proposed system is a real-time system. Experiments for validating the proposed system are conducted from the perspective of calibration, occlusion, accuracy, and efficiency. Furthermore, to demonstrate the practical performance of our system, a comparison of previously developed motion capture systems (the linear trilateration approach and the geometric trilateration approach) with the benchmark OptiTrack system is conducted, therein showing that the accuracy of our proposed system is $38.3\%$ and 24.1% better than the two aforementioned trilateration systems, respectively.
translated by 谷歌翻译